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ABSTRACT 



Decision analysis methodology was applied to the problem 
of some project management trade-off decisions required for 
the Patrol Frigate (PF) Land Based Test Sites (LETS) . In 
particular, the question of whether to install or simulate 
the Mark 92 mod 2 fire control system, the AN/SPS-49 radar 
and LM 2500 gas turbine engine at the LBTS was studied. The 
necessary decision analysis methodology required to 
formulate and solve the problem was presented. The results 
of this thesis indicated that the fire control system should 
be partially simulated, the radar should not be installed 
and the gas turbine engine should be simulated at their 
respective Land Based Test Sites. 
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I 



INTRODUCTION 



There is general recognition in the armed services that 
the management of projects concerned with weapons systems 
development and production is an increasingly complex and 
important task. The development and procurement costs of 
major weapons systems have steadily increased over the past 
ten years [Ref. 1]. Additionally, a need to reduce 
Department of Defense (DOD) spending has been identified 
many times by the President and other government officials 
as well as concerned citizens. These problems are placing 
an increasing burden on project management. The existence 
of cost and schedule over-runs and performance, 
maintainability, and reliability problems with the final 
weapons system hardware indicates there is room for 
improvement. 

There are several areas in project management where 
formal analysis can be of potential benefit to the project 
manager, to the Navy and to the researcher. One such area 
concerns how trade-off decisions are made among various 
factors such as cost, performance, and schedule. A good 
manager must have the ability to make reasonable judgments 
in these areas. He must determine the realizable 
alternatives and then select that alternative which best 
satisfies the overall objectives of the project. These 
judgments must often be made in highly uncertain 
environments . 

An approach to aid the manager in trade-off decisions, 
not typically used in the past, is the use of decision 
analysis. Decision analysis is a systematic procedure to 
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logically describe, specify and balance all factors which 
affect a decision. Among the factors considered in decision 
analysis is the probability that a specific outcome will 
occur from the various possible consequences. The preferred 
alternative also depends on the decision-maker's preferences 
for those consequences. The techniques used in decision 
analysis may enable a project manager to break a relatively 
complex judgmental situation down to a series of simple 
decisions. 



A. PURPOSES 

The purposes of this thesis were fourfold. The first 
purpose was to demonstrate that decision analysis techniques 
can be used as a project management tool. The second 
purpose was to provide the author with experience in 
conducting decision analysis in the project management 
environment. This was particularly useful because the 

research contained cases of multidimensional consequences 
under uncertainty. The third purpose was to provide 

examples of project management decision problems which could 
be incorporated into the United States Naval Postgraduate 
School's Adminstrative Science Curriculum. The forth 
purpose was to generate interest in the use of decision 

analysis in project management decision-making. 

B. LAND EASED TEST SITE TRADE-OFFS 

A current example of project management trade-off 
decision making is the problem of what constitutes an 
optimum mix of actual and simulated subsystems of the Patrol 
Frigate (PF) escort vessel which would be installed and 
tested at a land based test site. The testing must be 
satisfactory for PF production approval by the Defense 



System Acquisition Review Council (DS ARC) . Consequently 
three subsystems of the Patrol Frigate, the AN/SPS-49 radar, 
the Mark 92 mod 2 Fire Control System (FCS) , and the LM 2500 
gas turbine engine, were selected as examples of how 
decision analysis can be demonstrated in this thesis. Four 
alternatives were considered. They are 

(D installation of the actual equipment, 

(2) installation of some actual and some simulated 
equipment, 

(3) installation of entirely simulated equipment, or 

(4) no installation necessary at the land based test 
site. 



C. LIMITATIONS 



There are some limitations to this thesis. It was 
written in the decision environment of the project manager. 
External influences such as political, social and cultural 
factors were not included. Some generalizations were made 
in order to maintain this thesis unclassified. Cost and 
schedule parameters represent an order of magnitude and are 
the results of experienced judgments of personnel from the 
PF Ships Acquisition Project Office (PMS-399) located at the 
Naval Sea Systems Command Headquarters. A set of utility 
functions was developed for one individual concerned with 
the problem to illustrate the method and use of decision 
analysis. Finally, the preferences for consequences and the 
assessments of judgmental probabilities were provided by 
personnel in the PF Project Office. No refinements were 
made on these assessments from other sources. 



12 



D 



THESIS ORGANIZATION 



this thesis is organized in the following manner. 
Chapter II presents a background of the Patrol Frigate Land 
Based Test Sites (LETS) . Decision analysis methodology is 
explained in Chapter III. This explanation includes 
discussion of the systematic steps used in formal decision 
analysis and the evaluation of multidimensional consequences 
under uncertainty. Formulation of the decision analysis 
procedures used in deciding the alternative that best 
satisfies the objectives of LBTS testing for the three PF 
subsystems under consideration is described in Chapter IV. 
Chapter V provides the solution procedures and the 
recommended courses of action. Chapter VI summarizes the 
thesis, presents recommendations and provides suggestions 
for further research. 
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II. BA CKG ROUND 



On 13 July 1971, Deputy Secretary of Defense David 
Packard created a major change in the DOD management of 
major defense system acquisitions with the issuance of 
Department of Defense Directive 5000.1. The Directive, 
titled " Acquisition of Major Defense Systems," applies to 
programs designated as major by the Secretary or Deputy 
Secretary of Defense. This Directive recognizes that 
sucessful development, production and deployment of major 
defense systems are dependent on people, priorities and 
clearly defined responsibility. To this end, the policy 
invoked by this document designates the program manager as 
the key individual accountable for a major acquisition 
program. He should have a charter with sufficient authority 
to accomplish his program objectives and sufficient tenure 
to accomplish his task. The layers of management between 
the program manager and his service should be minimized 
£ Ref. 2]. 

A. PF PROJECT 

If a program r esults . in an expenditure in excess of 50 
million dollars for research and development, or is in 
excess of 200 million dollars during production or is urgent 
to National Security it will be classified as a major 
acquisition program. The Patrol Frigate Project has been 
classified in this category. 
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1 • Obje ctive 



The objective of the PF Project is to acquire by the 
late 1970's a class of ships which will provide, at least 
cost, the maximum improvement in the Navy's surface 
combatant capability to defend non-carrier forces against 
airborne and sub-surface attacks. 

The PF is an austere escort ship designed to provide 
maximum mission capability within the constraints of average 
follow ship cost of 47.7 million dollars (FY 73 dollars), a 
full load displacement not exceeding 3500 tons and total 
personnel accomodations not exceeding 185. Follow ship cost 
is defined to mean the unit production cost of ships built 
after construction of the lead ship. 

2. The D SA RC Rev i ew P r oc ess 

There are four life cycle phases for a normal weapon 
system acquisition program. The first phase is the 
conceptual phase. The objective of this phase is to define 
the operational need. The system concepts which warrant 
development are explored. The second phase is the 
validation phase in which alternative system concepts are 
validated as a basis for determining whether or net to 
proceed into full-scale development. The third phase is 
full-scale development. The objective of this phase is to 
design the system, construct a full-scale prototype for test 
and evaluation and provide the documentation needed to 
produce the system. The final phase is the production phase 
in which the system is produced for operational use. Before 
one phase is permitted to proceed into the next phase, 
approval must be obtained from the Defense System 
Acquisition Review Council. 
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The DSARC review process occurs at three decision 
point milestones during the normal life cycle of the system 
development. These points in time are defined as 

(a) DSARC I, which occurs between the conceptual and 
validation phase, 

(b) DSARC II, which occurs between the validation 
and the full-scale development phase and 

(c) DSARC III, which occurs between the full-scale 
development and production phase. 

Eefore a project can sucessfully pass the three 
DSARC decisions, it must be increasingly better defined and 
have minimized the technical uncertainty and program risks. 
Test and evaluation, to the extent it can be performed in 
each phase, is an important means for reducing such risk and 
uncertainty. 

3 • Test a nd E va luatio n 

Department of Defense Directive 5000.3 [Ref. 3] 
establishes the policy for the conduct of test and 
evaluation by the DOD components in the acquisition of 
defense systems. The general policy is that test and 
evaluation shall be commenced as early as possible and 
conducted throughout the acquisition cycle to assist in 
progressively reducing risks and assessing military worth. 

The Directive defines Development Test and 
Evaluation (DT6E) as the test and evaluation conducted to 
demonstrate that the engineering design and development 
process is complete and that the design risks have been 
minimized. DTSE commences as early in the development phase 
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as possible after DSARC II to demonstrate that technical 
risks have been identified and that solutions are in hand. 
Prior to DSARC III, the DT&E accomplished should ensure that 
the engineering is essentially complete and that all 
significant design problems have been identified and have 
solutions. 

Operational Test and Evaluation (OTGE) is conducted 
to evaluate the developed system's operational 
effectiveness. The Directive specifies that acquisition 
programs will be structured so that at least an initial 
phase of Operational Test and Evaluation (IOT&E) will be 
accomplished prior to DSARC III adequate to provide a valid 
estimate of the expected system operational effectiveness 
and suitability. 

Department of Defense Directive 5000.3 further adds: 



"The long, design, engineering, and 
construction period of a major ship will normally 
preclude completion of the lead ship and 
accomplishment of test therefore prior to the 
decision to proceed with follow ships. In lieu 
thereof, sucessive phases of Development Test and 
Evaluation (DT&E) and Operational Test and 
Evaluation (OT&E) will be accomplished as early as 
practicable at test installations and on the lead 
ship so as to rapidly reduce risks and thereby 
minimize the need for modification to follow ships." 



4 . PF Land Bas e d Test Site R ati onale 



The DSARC II review for the Patrol Frigate Program 
was held on 31 August 1972. The Deputy Secretary of Defense 
issued a Decision Memorandum for the Secretary of the Navy. 
In this Memorandum, one of the decisions reached by the 
Deputy Secretary of Defense was that approval of follow ship 
production should be contingent upon adequate test and 
evaluation individually on subsystems ana collectively at 
land based test sites f. Ref. 4]. In order to satisfy the 
requirements contained in Refs. 2-4 described above, the 
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Project Manager has • determined that the propulsion and 
combat systems for the Patrol Frigate should be prototyped 
at land based test sites (LBTS). 

The Propulsion System LBTS is now under construction 
at the Naval Shipyard in Philadelphia, Pennsylvania. The 
LBTS will consist of two LM 2500 gas turbine engines, one 
reduction gear complete with high speed clutches and brakes, 
one controllable reversible pitch propeller hub with dummy 
blades, one shaft, a control system and a control station. 
Figure 1 illustrates the Propulsion system arrangement. 

The Combat System LBTS, is located at MacArthur 
Field, Long Island, New York. Actual eguipment to be 
installed consist of one AN/SPS-49 air search radar, one 
AN/SPS-55 surface search radar, a Mark 92 mod 2 Fire Control 
System, two AN/UYK-7 computers used as Weapons Control and 
Support Processors and miscellaneous switchboards and 
communications eguipment. Functional simulators will be 
provided for the missiles, launchers, gun mounts, sonar and 
electronics countermeasure eguipment [Ref. 5]. Figure 2 
illustrates a block diagram arrangement of the Combat System 
LBTS. It is to be noted that the location of the Combat 
System LBTS is five miles from the coast line, surrounded by 
trees and dwellings. This will limit low flying aircraft 
operations and prohibit the actual firings of guns or 
missiles [Ref. 6]. 
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PF PROPULSION SYSTEM LBTS EQUIPMENT CONFIGURATION 



Note: 




FIGURE 1 
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PF COMBAT SYSTEM LBTS EQUIPMENT CONFIGURATION 




FIGURE 2 




B 



A IBIS PROBLEM 



The PF Development Concept Paper [Ref. 7] outlines the 
following critical test and evaluation questions and issues: 

(1) Are the individual combat subsystems to be 
incorporated in the ship technically acceptable and 
operationally suitable and effective? 

(2) Does the integrated propulsion system installed at 
the Propulsion System L3TS demonstrate sufficient 
performance and reliability to indicate that operational 
requirements will be met? 

(3) Is the integration of the various Combat Systems 
through their respective interfaces adequate, in terms of 
data transfer characteristics such as handling capacity, 
rate and guality, to meet operational . requirements? 

(<») Do those operational characteristics of the 
integrated Combat and Propulsion Systems, including 
reliability and maintainability, which can be estimated 
based on IOT&E, show a reasonable probability that the ship 
class when afloat will be able to satisfactorily accomplish 
the mission for which it was designed? 

One of the decision problems, then, for the PF Project 
Manager was what the composition of subsystems should be in 
the Combat and Propulsion Systems LBTS so that "adequate 
test and evaluation" [Ref. 4] with satisfactory results can 
be supplied to DS ARC III, scheduled for May 1975. The 
installation and testing of these PF subsystems are to be 
accomplished to ensure that the critical test and evaluation 
guestions outlined in the Development Concept Paper [Ref. 7] 
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are answered at minimum cost. Additionally, the primary 
objective of the LBTS after DSARC III production approval 
will be to verify operating, maintenance and test 
procedures, to provide operational training and to test 
system upgrading [Ref. 8], 

the Project Manager was faced with a complex problem to 
solve. The problem contains the multiple measures of 
effectiveness of cost, schedule and future usefulness under 
conseguences of varying degrees of uncertainty. It is this 
problem which was used in Chapters IV and V to illustrate a 
decision analysis approach to project management trade-off 
situations. Chapter III contains a description of the 
methodology necessary to formulate and solve the problem. 
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Ill 



• DECISION_ANALY^S_METHODOLOGY 



Man is often confronted with situations in which the 
consequences of any action he takes are not certain. Events 
may intervene which he can not control or predict with 
certainty. A large number of decisions under uncertainty 
are made by intuition [Ref. 9]. The intuitive decision 
process is accomplished in the decision-maker's mind. 
Because of this, there is no way to verify that this type of 
decision is the logical consequence of the choices, 
information and preferences that were available to the 
decision-maker. For many problems, however, it is important 
that the decision-maker is able to show people why he 
arrived at a particular decision and also for them to be 
able to see what changes in factors surrounding that 
decision might have led to a different decision [Ref. 9]. 
Another characteristic of the intuitive decision process is 
the human tendency to equate the quality of the decision 
with the quality of the outcome it produces. For example, 
consider a situation where an investor decides to buy some 
new stock. If he loses money, the tendency is to say that 
the investor made a bad decision; conversely-, if he makes 
money, then he made a good decision. A good decision is a 
decision which maximizes- the probability of a good outcome; 
hence, making a good decision is no guarantee of a good 
outcome. The decision-maker has control of the decision. 
He does net have control of the outcome. 

The purpose of decision analysis is to allow the 
decision-maker to make consistent good decisions and to 
formulate them in quantitative terms that can be conveyed 
from one person to another. Formal decision analysis is a 
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systematic process comprising the following steps: 

(1) structuring the problem, 

(2) assessing relative preferences for possible 
conseguences, 

(3) evaluating the probabilities for uncertainties and 

(4) determining the best course of action from the 
information in the preceeding steps [Ref. 10]. 

This process is an iterative process. First, a broad 
description of the problem with rough assessments of the 
preferences for the conseguences and probabilities fcr the 
uncertainties is analyzed. On the basis of the first 
analysis, alternatives are added or removed from 
consideration. The measurements are refined and the process 
is repeated until there is satisfaction with the results of 
the analysis [Ref. 11]. 

The purpose of this chapter is to acquaint those 
unfamiliar with decision analysis with its theory and 
techniques to the extent necessary to formulate and solve 
the PF Land Eased Test Site problem introduced in Chapter 
II. This chapter is organized to explain the methodology of 
decision analysis for each step in the formal analysis. 
Before proceeding, it is necessary to explain certain terms 
and notations which are used throughout the remaining parts 
of this thesis. 



A. CLARIFICATION OF TERMS AND NOTATIONS 



The terras "is indifferent to", "is preferred to", 
"lottery" and "utility function", are widely used in the 
following sections of this thesis. For clarity, they need 
to be explained. The term "is indifferent to" is to be used 
to mean the same as the statement "the decision-maker is 
indifferent to receiving either of the outcomes." The term 
"A is preferred to B" is to be used to mean the same as the 
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statement "the decision-maker prefers A over B." 



The term "lottery" is defined as a gamble of some 

* 

uncertain event E where the prize X is won if the event E 

occurs and the prize X is won if the event E does not 

* 

* 

occur. Let p represent the probability that E occurs and 
* 

let 1 - p represent the probability that E does not occur. 

Notationally , the lottery L will be represented as 

E 

<X ,p ,X >. 

❖ 



The term "utility function" is defined as a function u 
which assigns a real value to every consequence a and b such 
that u(a) is larger than u (b) if ana only if a is preferred 
to b [Eef. 10]. The notation u (a) is expressed as the 
"utility of consequence a" and is represented by a real 
number. 

With the above terms clarified, the steps in a formal 
decision analysis process can be explained. The first step 
in this process is structuring of the problem. 

B. PROBLEM STRUCTURE 

In structuring a problem in which events are uncertain, 
the options or alternatives are enumerated. Next, all the 
events that can possibly occur are specified. As a last 
step, the alternatives and uncertain events are arranged in 
chronological order. 



25 



A type of diagram known as a decision-flow diagram or 
"tree" is a useful tool in decision analysis. It is a 
chronological arrangement of the alternatives which are 
controlled by the decision-maker and the events determined 
by chance [Ref. 11]. To illustrate the construction of a 
decision-flow diagram, consider the following problem. A 
decision-maker is faced with two alternatives, I and II. 
Both alternatives involve a situation where the outcomes a 
or b are uncertain. If a occurs, then the decision-maker 
must decide between alternatives III and IV. Alternative 
III also involves an uncertain situation leading to either 
the outcome c or d . 

The decision-flow diagram is shown in Figure 3. Observe 
that the branching points or forks are of two types: 
decision forks and chance forks. A decision fork is 
designated by a small square and a chance fork by a small 
circle. There is additional information provided in the 
diagram which will be discussed in the sections below. 

With the alternatives and uncertain events described by 
a decision-f low diagram, the next step in the decision 
analysis . process is the assessment of the relative 
preferences for the consequences. 



C. ESTABLISHMENT OF PREFERENCES 



The establishment of preferences for the consequences 
provides the decision-maker with the basis for the rational 
choice between the alternatives. This depends upon the 
views and attitudes of the decision-maker. The consequences 
may encompass a number of factors or attributes such as 
cost, schedule and performance. These attributes might also 
be of an intangible nature such as. goodwill, morale and 
politics . 
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EXAMPLE DECISION-FLOW DIAGRAM 



UTILITY 




(X ,x ,x ) =u 
12 3 1 



<VW =U 



2 



‘VVV's 



(x ,x ,x ) =u 

1 2 3 4 



(X ,x ,x ) =u 
1 2 3 5 



(X ,X ,X ) =u 
12 3 6 





X ) =u 
3 7 



(x , x , x ) =u 
12 3 8 



0: DECISION FORK 

Q: CHANCE FORK 

FIGURE 3 
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In this step in the decision analysis process, an 
objective function is defined to indicate a measure for the 
preferences for the conseguences. 

A general methodology for defining an objective function 
in decision analysis problems exists in the form of utility 
theory. In this thesis no attempt is made to develop the 
theory in detail. It will be developed only to the extent 
necessary to formulate and solve the PF LETS problem. The 
development draws heavily upon work by Giaugue [Ref. 12] and 
Keeney [ Ref . 13 ]. 

Conseguences may be described by a single attribute or a 
multiple set of attributes. Both situations are applicable 
to the analysis in this thesis and are separately presented 
below. 

1 • Single Att r ib u tes 



In the case of a single attribute, an objective 
function, hereafter called a utility function, can be 
defined which has the property that the maximum expected 
utility among the alternatives indicates the most preferred 
action [ Ref. 12 ]. 



A utility function with a single attribute can be 

* 

constructed in the following manner. Define X and X as 

* 

the upper and lower limits over a range of possible 



conseguences X such 

i 

conseguence X , define 
i 



that X >X >X . 

i * 

the utility u(X ) 

i 



For every possible 
as the value p 

i 
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such that the decision- maker is indifferent to receiving X 

i 

* 

for certain and receiving the lottery <X ,p ,X >. The value 

i * 

* 

of p ranges from zero to one, where by convention, u (X ) 
i 

equals one and u (X ) equals zero [Ref. 12]. 

* 



Once a set of points (X ,p ) have been established, 

i i 

a utility curve may be drawn. Figure 4 illustrates . three 

possible utility curves. A utility curve generally has two 
characteristics. It is smooth and the general shape of the 
curve is either convex, straight or concave as illustrated 
respectively by curves 1, 2 and 3 of Figure 4. Any break in 
the curve would indicate either an inconsistency in the 

* 

choices for p in the lottery <X ,p ,X > used to assess the 
i i * 

points of the curve, or a quantum jump in preference for a 

small change in X . A convex curve indicates a risk averse 

i 

behavior [Ref. 11]. That is, the decision-maker is more 

inclined to take a consequence known for sure than to take a 
gamble with the same expected value. A concave curve 
indicates that the decision-maker is risk seeking. He is 
more inclined to take the gamble than to take the known 

consequence, A straight line indicates that the 

decision-maker acts on the expected value of the consequence 
[Ref. 11]. He is neither risk averse nor risk seeking. 

Once the utility curve is established for a single 
attribute consequence, a value from one to zero is assigned 
to each consequence corresponding to the point on the curve. 
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A higher value for a consequence indicates greater 
preference for that consequence than for a consequence with 
a lower utility value. Reference 14 contains detailed 
treatment of utility functions with single attributes. 



EXAMPLES OF UTILITY CURVES 




X X 

♦ 



Consequence X 

i 

FIGURE 4 



2. Multiple Attri but es 



The basic concept of the construction of a utility 
function with a single attribute described above can be 
generalized to the case where many attributes must be 
considered. However, the above assessment scheme is 
impractical. First, too many points must be assessed. 
Secondly, humans find it difficult to think in terras of 
multiple attributes. In decision problems under 

uncertainty, many people when faced with situations where 
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more than one attribute is relevant, tend to pick the one 
attribute judged most important to them and then make the 
decision on that factor alone [Ref. 10]. 

There are procedures for decomposing a multiple 
attributed utility function into combinations of 
unidemensional functions. Conditions required for 
decomposition include the properties of utility 
independence, pairwise preferential independence and 
pairwise marginality. These are described below. 



Keeney [Ref. 13] shows that a multia ttributed 
utility function can be expressed in one of two forms, 
additive or multiplicative, dependent on which of the 
properties of utility independence, pairwise preferential 
independence or pairwise marginality hold. If a utility 
function of multiple attributes can be expressed in these 
forms, then the task of defining the utility function is 

much easier. Suppose X = (x ,...,x ) describes a 

1 n 

consequence where u (X) denotes the utility of the 

consequence X. Utility independence is defined in the 

following manner. Let x = (x ,..,x ,x ,...,x ). The 

i 1 i-1 i+1 n 

attribute x is utility independent of x _ if- the 

i i 

decision-maker's relative preference for x , with x _ held 

i i 

fixed, is the same regardless of the actual value of x 

i 

chosen. Order one mutual utility independence is defined to 

mean that x is utility independent of x _ for all i. If 
i i 

order one mutual utility independence holds then u (X) can be 
expressed in the quasi-additive form 



u (x 



x ) = u (x. ) + ? .5 C .u. (X . ) U (X ) + .... 

n i=1 l i i=1]-1 i] l l j ] 
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Pairwise preferential independence is said to hold 
if the trade-offs one is willing to make between attributes 
taken two at a time, are not dependent on the values of the 
remaining attributes. Let X _ = 

ij 

(x ,..,x ,x ,..,x ,x ,...,x ), and let x be a 

1 i-1 i+1 j-1 j+1 n ij” 

particular value from X The attributes x , x are 

ij i j 

pairwise preferentially independent of X _ if one's 

ij 

preference order for the consequences (x , x , x _) with 

i j ij 

x _ held fixed, does not depend on the particular value 

ij 

x [Ref. 13]. 
ij” 



If for any pair of attributes x and x , the lottery 

i j 

0 0 

< (x ,x ),0.5,(x ,x )> is indifferent to the lottery 

i j i j 

0 0 

< (x ,x ),0.5,(x ,x )> then pairwise marginality is said to 

i j i j 

hold [Ref. 12]. 



With the ideas of utility independence and pairwise 

preferential independence presented, Keeney's results can be 

more precisely stated [Ref. 13]. Let X = (x ,...,x ) be as 

1 n 

previously defined, with n > 3. If, for some x , x and x 

i . i j 

are pairwise preferentially independent of 



(x ,..,x ,x ,..,x ,x ,...,x ) for all j t i and x 
1 i-1 i+1 j-1 j+1 n i 

is utility independent of x_, then either 



U(X) = i k u. (X.) (1) 

3=1 1 1 1 



32 



or 



1 + K u (X) = TT [ 1 + Kk u (x ) ] (2) 

i=1 i i i 

where u and u are utility functions scaled from zero to 

i 

one, the k are scaling constants with 0 < k <1 and K > -1 
i i ' 

is a non-'zero scaling constant. Equation (1) is the 

additive form and equation (2) is the multiplicative form. 

Given that the conditions of Keeney^s Theorem hold, 
he provides a property required to show whether the function 
is additive (1) or multiplicative (2) . He shows that if 
pairwise marginality holds then the function must be 
additive; otherwise, it is multiplicative. Tafcle I 

summarizes the properties necessary for each simplification 
[Ref. 12]. 



TABLE I 

UTILITY FUNCTION SIMPLIFICATION 



Simplification 


Properties 




1st order 


Pairwise 






utility 


pref erentia 1 


Pairwise 




indep. 


indep. 


marginality 


Qua si additive form 


X 






Multiplicative form 


X 


X 




Additive form 


X 


X 


X 



Referring to Figure 3, there are three attributes 
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x , x and x which describe each outcome of the tree. For 
1 2 3 

illustration, the following utility function might be used : 

u(x ,X ,X ) = U(X ) + U (X ) + u(x ) = u . 

1 2 3 1 2 3 i 

This utility function, in the additive form, maps the 
consequences x^, x^ and x^ into a scalar value indicated by 

u , where i = 1,...,8, at each branch-tip of the tree, 
i 

D. JUDGMENTAL PROBABILITIES 

The decision-flow diagram is one of the decision 
analysis methods used in structuring a problem. Utility 
functions can be used for the assignment of preferences for 
the consequences of the outcomes at each tip of the tree. 
What remains to complete the information included on the 
decision-flow diagram is the assignment of the judgmental 
probabilities at the chance forks representing the uncertain 
events. This is the third step in the decision analysis 
process. 

Raiffa [Ref. 11] addresses the question of whether the 
decision-maker's hunches or vague impressions should be 
calibrated, and if so, how this should enter into the formal 
decision analysis process. He argues that if a 
decision-maker wishes to act consistently, then he cught to 
assign values to judgmental probabilities such that the sum 
of the probabilities of an event occurring and not occurring 
equals cne. This judgmental probability assessment for an 
event should not depend on the outcomes. He points out that 
judgmental probabilities satisfy the usual rules of 
probability theory and can be used in the same manner as 
objective probabilities. 
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Judgmental probabilities are used as a measure of the 
decision-maker's beliefs concerning the uncertainty of an 
event occurring, provided that these beliefs are 

consistently applied to every uncertain event in the 
analysis. They are assigned to each chance fork of the 
tree. In Figure 3, they are represented as p^, 1-p p^, 

1-p , p , 1-p , p and 1-p . With this information, the 
2 3 3 4 4 

final step in any iteration of the decision analysis process 
is to determine the recommended course of action. 



E. RECOMMENDED COURSE OF ACTION 



Determination of the recommended course of action 
involves a sequence of calculations called by Raiffa [Ref. 
11] the "averaging out and folding back" procedure. This 
procedure is often referred to as the process of backwards 
induction in the theory of dynamic programming [Ref. 11 ]. 
The procedure starts at the tips of the tree and consists of 
computing the expected utility of each chance fork and the 
selection of the greatest utility at each decision fork. 
The process is repeated for each level of the tree until the 
starting decision fork is reached. The alternative with the 
greatest expected utility is selected as the recommended 
course of action. The selection of the maximum expected 
utility is an appropriate means of determining actions 
consistent with the decision-maker's attitudes and opinions 
[Ref.. 12]. This point is presented and developed in such 
sources as Schlaifer [Ref. 16] and Pratt, Raiffa and 
Schlaifer [ Ref. 14 ]. 

To illustrate the "averaging out and folding back" 
process, the information contained in Figure 3 is used. 
Starting at the chance fork labeled (T) , the expected 
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utility is computed as 



u p + u 
1 3 2 



<1-P 3 ) 



E 

1 



Moving backwards in the tree, 
decision fork, labeled [T] 



the next fork encountered is 

The value of E or u 

1 



a 

3' 



whichever is greater, is selected. 



For illustration. 



E is 

1 



selected. Continuing backwards through the tree, a chance 



fork, labeled (3), is encountered. At this point, the 
expected utility of the chance fork is computed as 



E 

1 



+ u 

4 






E . 

2 



Alternative I has now been reached and the expected utility 

of this alternative is E . In similar fashion, the expected 

2 

utility cf alternative II is computed. The results are 
compared and the alternative with the greatest expected 
utility is selected as the recommended course of action. 



F. SUMMARY OF CHAPTER 

On most occasions, people make decisions intuitively and 
more or less inconsistently. There are occasions when the 
decision must be made in a reasoned, deliberate manner. 
Decision analysis methodology was introduced to provide 
this. In the systematic process of decisicn analysis, the 
decision-maker starts by structuring the anatomy of his 
problem in a decision- flow diagram that depicts the 
chronological interactions between his alternatives at any 
stage and the events which are controlled by uncertainty. He 
scales his preferences for the consequences at the tips of 
the decision tree in terms of utility values and scales his 
judgments about uncertain events in terras of probability 
assignments at the chance forks in a consistent manner. 
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Finally, he selects his best strategy for action by the 
process of " averaging out and folding back." 

Host "real life" problems are complex. Trees exhibiting 
the structure of these problems can be so complex as tc make 
adetailed analysis of the alternatives impractical. There 
is an "art" to analyzing real problems as described by 
Baiffa [Bef. 11]. An iterative process is used. Initially, 
alternatives and measurements are specified in a rough 
manner. Freguently, some of the decision branches will turn 
out to be nonoptimal and can be eliminated from the tree. 
If the decision branch is close to the base cf the tree, a 
sizable portion of the tree can be eliminated. After 
elimination, further effort can be put into refining the 
description of the remainder cf the tree. This iterative 
process is repeated until the decision can be satisfactorily 
made. 



The necessary decision analysis methodology has now been 
provided. Formulation and solution of the PF Land Based 
Test Site problems are presented in the following two 
chapters . 
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IV. PF LBTS PROBLEM FORMULATION 



Chapter II presented the background of the PF Land Based 
Test Sites and developed the trade-off problem for the 
Program Manager as to what would be the necessary mix of PF 
subsystems which would be installed at the LBTS to insure 
adeguate testing for the DSARC III review process. The 
Project Manager has to make sure the testing of the 
subsystems installed at the LBTS provide a valid estimate of 
the expected system operational effectiveness and 
suitability. 

The Development Concept Paper [Ref. 7] lists in the Risk 
Watch List the PF subsystems which are considered to have 
some degree of development risk. Among these items are the 
Mark 92/2 fire control system, the AN/SPS-49 radar and the 
LM 2500 gas turbine engine. Individual Technical and 
Operational Evaluation 'are to be conducted on the subsystems 
listed [Ref. 5], but some seemed candidates for the Land 
Based Test Sites including the fire control system, radar 
and engine. In this chapter, the options for these three 
subsystems are examined more closely using the tools 
developed in Chapter III. This examination comprises an 
initial analysis proceeding in the decision analysis steps 
previously described. Considerable data for this analysis 
was provided the author by the PF Test and Evaluation 
Manager and the Assistant Test and Evaluation Manager during 
interviews with them at the PF Project Office, by 
correspondence and by telephone interview. 

The first step in the decision analysis process is to 
structure the problem in terms of a decision tree. The 
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formulation of the problem structure is described below 



A. PROBLEM STRUCTURE 

In testing of the Mark 92/2 FCS, SPS-49 radar and LM 
2500 gas turbine engine, the Project Manager is concerned 
with the installation and testing costs, the schedule, the 
usefulness of the subsystems to the PF Project at the LBTS 
in the future, and the uncertainty of passing DSARC III. 
His options for all three subsystems, in general, are: 

(D install actual or simulated equipment at the LETS, 

(2) conduct at-sea testing of each subsystem during 
Operational Evaluations, 

(3) monitor tests at the vendor's activity or at sea or 

(4) conduct no further testing. 

To structure the LBTS problem, the options or 
alternatives are enumerated. Next, all the events that can 
affect the installation and testing of the subsystems are 
listed and arranged in chronological order. Lastly, a 
decision tree is constructed. 

An intuitive process, in considering the options, was 
suggested by the PF Test and Evaluation Manager. This 
process is shown as a logic-flow diagram in Figure 5. To 
enumerate the alternatives, the options displayed in the 
"LBTS" boxes of the logic-flow diagram are examined. 
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INTUITIVE LOGIC-FLOW DIAGRAM 



PF LBTS PROBLEM 




FIGURE 5 
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1 • Alt er natives 



Basically, there are four alternatives considered. 
The first alternative is to install the actual subsystem in 
its entirety. The second alternative is to install a 
completely simulated subsystem. The third alternative is to 
install a partially simulated subsystem. The fourth 
alternative is not to install the subsystem at the LBTS. 

The above alternatives apply for the installation 
and testing problem of the Hark 92/2 FCS. They dc not 
entirely apply for the cases of the SPS-49 radar or LM 2500 
gas turbine. The alternative of partially simulating some 
components for the SPS-49 radar system was not considered. 
It would be just as feasible to simulate the entire radar 
system as to simulate some components such as the antenna 
and transmitter. Therefore, three alternatives were chosen 
for the AN/SPS-49 radar problem. They are: install the 
actual eguipment of the radar, simulate the radar system or 
not install the radar at the LBTS. 

The alternative of partially simulating the gas 
turbine engine was not considered because it does not make 
sense to do so. The alternative of not installing the 
engine at the Propulsion LBTS was also not considered since 
the engines must be represented if there is to be a 
Propulsion LETS. The assumption is made that the Propulsion 
Syste.m LBTS is desirable and necessary for the development 
of the Patrol Frigate. For the LM 2500 gas turbine engine, 
two alternatives are considered. They are to install the 
actual engine or to simulate the engine at the LBTS. 

With the alternatives enumerated for the 
installation and testing of the three PF subsystems at the 
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LBTS, the events which affect them are specified and 
arranged in time seguence. 

2. Events 



The uncertainty of passing DSARC III scheduled for 
May 1975 is the primary concern of the PF Project Officer. 
The DSARC III review determines whether the PF development 
risks have been minimized so that the PF Acquisition Project 
may progress into the production phase. It is assumed that 
the Patrol Frigate is a necessary addition to the military 
posture for National Defense. If the DSARC III review does 
not consider the technical and project risks tc be 
minimized, DSARC III will be rescheduled after all the 
deficiencies have been corrected. It will be at this point 
where the decision tree structure will be cut-off in the 
initial analysis of this thesis. 

There are two events besides the DSARC III review 
which must be considered in structuring the decision tree. 
The first event is the LBTS Certification tentatively 
planned for January 1975. The second event is the 
Operational Assist which will be conducted by Commander 
Operational Test and Evaluation Force (COKOPTEVFOR) during 
the period February to Hay 1975 [Ref. 17]. 

LETS Certification will consist of a series of tests 
conducted by the LBTS contractor to verify that the Combat 
System subsystems and Propulsion System subsystems have been 
successfully integrated, are operational and are ready to be 
used for other demonstrations at their respective LBTS. 
Certification will be approved by the PF Project Manager. 
If the LBTS fails Certification, additional time and money 
will be used to correct the deficiencies noted in order to 
pass Certification. If funds are not sufficient to correct 
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the deficiencies, the LBTS will have to be cancelled. 

Another decision event occurs prior to 
Certification. If there is enough time to do Certification 
and Operational Assist, then Certification will commence as 
scheduled. If there is not enough time to do both, then the 
certif ication process will not be formally done and the 
Operational Assist will commence as scheduled [Ref. 17]. 

An Operational Assist project is assigned by the 
Chief of Naval Operations (CNO) in response to a favorable 
program initiation decision by the Secretary of Defense. 
For the case of the Patrol Frigate, it was assigned by Ref. 
4. The major purpose of an Operational Assist is to 
establish confidence in the program worth and readiness for 
the commitment of resources for full-scale development [Ref. 
18], A decision event occurs if the Operational Assist is 
determined not satisfactory by the COMOPTEVFOR 
representative. At this point in time, it must be 
determined whether to correct the items which are 
unsatisfactory or to cancel the LBTS. Table II summarizes 
these events and their consequences in time sequence. 

With the alternatives and the uncertain events 
arranged in chronological order, a decision tree is 
constructed to complete the structuring of the LBTS problem. 
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TABLE II 



UNCERTAIN EVENTS OF LBTS 



EVENT 


CONSEQUENCE 


TIME-F FAME 


1. Time available 
for certification 
and Op Assist 


If time available, 
do both; else, 
do Op Assist. 


Fall 1574 


2. Certification 


Pass or fail. If 
passed, do Op Assist. 
If failed, do over 
or cancel LBTS. 


January 1975 


3. Op Assist 


Pass or fail. If 
passed, do DSARC III. 
If failed, do over 
or cancel LBTS. 


Spring 1975 


4. DSARC III 


Pass or fail. If 
passed, analysis done. 
If failed, do over. 


May 1975 



3. Decisi on Tree s 



Figure 6 depicts- a typical branch of the decision 
tree for the alternatives of actual installation, partial 
installation or simulation of the subsystems at the LBTS. 
The first chance fork in the branch represents the 
uncertainty of the time available for Certification and 
Operational Assist. Only one alternative is selected at 
this point. If there is time available, then the 
alternative is to do both Certification and Operational 
Assist. If there is no time available for both tests, then 
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the alternative is to do Operational Assist only. The 
second chance fork represents the uncertainty of 
Certification. If it is passed# then the Operational Assist 
is conducted. If it is failed, then a decision is made to 
cancel the LBTS or do the Certification over. The third 
chance fork of the branch represents Operational Assist 
testing. If the Operational Assist is passed, then DSARC 
III commences as scheduled. If the Operational Assist is 
failed, then a decision is made to cancel the LBTS or do the 
Operational Assist over at a later date. The final chance 
fork is the DSARC III review. If it is passed, the most 
preferred corseguence is achieved. If it is failed, then a 
decision is made to either cancel the FF Project or to 
reschedule DSARC III at a later date. As was pointed out 
above, the assumption is that the Patrol Frigate is 
essential to the National Defense posture, so that, in the 
analysis, the "do over" branch will be weighted more 
favorably than the "cancel" branch. 

Figure 7 depicts the "no installation " branch of 
the decision tree for the Mark 92/2 FCS and SPS-49 radar. 
There is one chance fork representing DSARC III in which it 
is either passed or failed. 

The structuring of the Land Based Test Site problem 
is completed. The next step in the decision analysis 
process is to evaluate the conseguences of the decision tree 
in terms of utility numbers. This process is described 
below where a set of utility functions for one individual 
concerned with the problem is developed to illustrate the 
method and use of decision analysis. 
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INSTALL EQUIPMENT BRANCHES 




FIGURE 6 



TYPICAL LBTS DECISION TREE 
INSTALL NO EQUIPMENT BRANCH 




B. ASSIGNMENT OF UTILITY NUMBERS 

In order to provide a basis for the rational choice 
between the alternatives, utility functions are established 
to indicate a measure for the preferences of the 
consequences. The consequences for the LETS problem are 
described by the attributes of cost, schedule and future 
usefulness. A fourth attribute, performance, was a 
possibility. It is assumed that all three subsystems will 
have to meet some previously specified performance level; 
therefore, a performance attribute was not considered for 
this initial analysis. 

The attributes which describe the consequences are 
tangible and are measured in millions of dollars for cost, 
months for schedule and scalar values for usefulness. The 
scalar value for usefulness ranges from zero to three. The 
value zero is assigned for a subsystem which is not 
installed at the LBTS. The value one is assigned to a 
simulated subsystem. The value two is assigned to a 
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partially simulated subsystem and the value three is 
assigned to the installation of the actual subsystem. Table 
III lists the attribute ranges determined from interviews 
with the PF Test and Evaluation Manager and the Assistant 
Test and Evaluation Manager [Eefs. 17,19]. The values for 
cost and schedule have been generalized reflecting an order 
of magnitude only. 



TABLE III 



ATTRIBUTE RANGES 





COST 

(million) 


USEFULNESS 

(scalar) 


SCHEDULE 

(month) 


1. AN/SPS-49 








minimum 


1.5 


0. 


1 . 5 


maximum 


3.5 


3 


4.0 


2. MK 92/2 FCS 








minimum 


7.5 


0 


1.5 


maximum 


18. 5 


3 


6.0 


3. LM 25 CO GT 








minimum 


3.0 


0 


10.0 


maximum 


15.0 


3 


16.0 



Since a consequence for the LBTS problem is represented 
by multiple attributes, the utility function will be 
described, in some form, by a combination of the individual 
attribute utility functions. Once they are established, the 
utility function might be expressed in the additive or 
multiplicative forms exhibited in Chapter III dependent on 
the properties of the attributes. Establishment of the 
individual attribute utility functions is first discussed. 
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1 • Ind ividual At tribute Utilit y Func tions 

A discrete utility function consisting of four 
points was established for the usefulness attribute in the 
analysis of all three subsystems. A straight line utility 
function for the schedule attribute for the SPS-49 radar 
decision tree was derived and a family of four utility 
curves was established for the schedule attribute describing 
the conseguences of the fire control and gas turbine 
decision trees. A straight line utility function 
representing the cost attribute was derived for all three 
subsystem decision trees. Table IV summarizes the results. 
The method for arriving at these utility functions is 
discussed in the paragraphs below. 

The attribute of future usefulness consists of four 
integers, zero, one, two and three. The utility of the 
value zero is zero since it is least preferred and the 
utility of the value three is one since it is most 
preferred. To assess the utility of the values one and two, 
the PF Assistant Test and Evaluation Manager was asked the 
following guestions [Ref. 17]: 

•'If you are indifferent between the future 
usefulness value one and the lottery <3,p^,0>, what do you 

feel the probability p^ of getting a future usefulness value 

three is?" 



"If you are indifferent between the future 

usefulness value two and the lottery <3,p ,0>, what do you 

2 

feel the probability p^ of getting a future usefulness value 
three is?" 
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would be 



The reply to these questions was that p 

0.5 and p^ would be 0.85. With this information, the 

utility function for the attribute future usefulness was 

evaluated. Given four discrete values, 0, 1, 2 and 3, with 
u^ (0) = 0 and u^ (3) = 1, then - 

u (1) = 0.5 u (3) + 0.5 u (0) = 0.5 

1 1 1 

and 

u (2) = 0.85 u (3) + 0.15 u (0) = 0.85. 

2 11 



A utility measure for the schedule attribute for the 
three PF subsystems will not have similar worth. As was 
disclosed in conversations with the PF Project Assistant 
Test and Evaluation Manager [Ref. 20], the schedule for the 
AN/SPS-49 radar is not critical. In assessing a utility 
function for this subsystem, he would be indifferent to 
gambles with respect to the schedule. He also indicated for 
the Mark. 92/2 FCS and gas turbine engine, their schedules 
are very critical. If he is afforded a choice between a 
gamble consisting of uncertain schedules and a schedule 
which was known, then he would take the known schedule in 
every case, indicating a risk averse behavior. He added 
that as long as the installation and test schedule were met 
for DSARC. Ill, he would evaluate the utility of any schedule 
less than the maximum as one. 

The ranges for the maximum and minimum schedule for 
the radar are 1.5 and 4.0 months. Since the preferences for 
schedule uncertainties for the SPS-49 radar are not risk 
averse, a straight-line utility function can be used to 
express the expected value for the installation and testing 
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schedule 



The straight line connects the 



point 



(x =4 . 0, u (4.0) =0) 
2 2 



with the point 



(x ( 1 . 5) = 1 ) . 



Since 



the function represents the expected value of the SPS-49 

schedule, the equation of the utility function cf the 

attribute schedule, u (x ) , will be 

2 2 



x 

2 



u (x ) 1.5 + [1 - u (x ) ] 4.0 

2 2 *2 2 

4 - 2.5 u (x ) . 

2 2 



Hence, 



u (x ) = ( 4 T X J / 2 * 5 * 
2 2 2 



The utility function for the attribute schedule of 
the SPS-49 radar, is represented in the following graph: 

AN/SPS-49 Radar 




Schedule (months) 



The LM 2500 gas turbine engine schedule ranges from 
ten to sixteen months. The Mark 92/2 FCS schedule ranges 
from 1.5 to six months. Because the schedule is critical 
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there will be risk adversion 



for these two subsystems, 
towards any gambles of uncertain schedules. A utility value 
of one will be assigned to any schedule that is less than 
the planned schedule for DSARC III. With this information, 
the utility function curve for the fire control system would 
be similar to one of the curves in the following graph. 
Four utility values, as indicated on the graph, will be used 
to determine the sensitivity of the results. 



MK 92/2 FCS 




The curve for the utility function of the gas 
turbine schedule would similarly look like one of the curves 
in the next graph. 



Utility 




Schedule (months) 



Values for the utility function for schedule for the engine 
and fire control system will be obtained from each of the 
four curves of the two graphs above and used in the solution 
of the decision trees to determine the sensitivity of the 
results. If there is a significant difference in the 
solution results between these values, then further research 
in the development of the utility function will be required. 

Ihe evaluation of the utility function for the 
attribute cost was next considered. The installation and 
testing costs for the SPS-49 radar range from 1.5 to 3.5 
million dollars. The LM 2500 gas turbine engine costs range 
from three to fifteen million dollars. The Mark 92/2 FCS 
costs range from 7.5 to 18.5 million dollars. The minimum 
costs represent the best costs for installation and testing, 
if all events are passed on schedule. The maximum costs 
represent the worst costs as if all events are failed and if 
no more funds are available to correct deficiencies. 

It seemed plausible that one utility function for 
the three subsystems could be derived where the costs range 
from zero to 18.5 million dollars. From discussions with 
the Assistant Test and Evaluation Manager [Ref. 20], this 
seems to be a valid assumption. • He is willing to take 
gambles on the basis of the expected cost values over the 
entire range for the three subsystems under consideration. 
This is justified because a vast range of assets is 
considered. A decision-maker, considering the vast amounts 
of funds available, would not be necessarily risk averse. 
It is realized that the cost decisions for each subsystem 
would be made over a much smaller range as indicated for 
each PF subsystem. 

The utility function for costs was derived as a 
straight line connecting the point (x^ = $ 1 8 . 5M, u ^ ( $ 1 8 . 5M) =0) 
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The equation cf the 



with the point lx =.£0M,u 

3 3 

utility function of cost, u 



($0M) =1) . 
(x^) , is 



* = u (x ) 0 + [1 - u (x ) ] 18.5 

3 3 3 3 3 

= 18.5 - 18.5 u (x ). 

3 3 



Hence, 



u ( x ) = (18.5 - x ) / 18. 5 
3 3 3 



The utility function for the cost attribute is 
represented in the following graph. 




Cost (million) 



With the individual attribute utility functions 
specified, the utility functions for the fire control, radar 
and gas turbine engine consequences can be represented as a 
combination cf the three individual utility functions. This 
utility function may be expressed in the multiplicative or 
additive form dependent upon which of the attribute 
properties of utility independence, pairwise preferential 
independence or pairwise marginality hold. The next section 
is devoted to the verification of which of these attribute 
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properties hold. 



TABLE IV 

INDIVIDUAL UTILITY FUNCTIONS 



Attribute 


Applicable to 


Utility Function 


1. Future usefulness; 


All 


u (0) =0.0 

1 

u (1) = 0.5 
1 

u (2) =0.85 

1 

u (3) = 1.0 


2. Schedule; 


SPS-49 radar 


u {x ) = (4-x )/2.5 
2 2 2 




MK 92/2 FCS 


Family of curves 




Gas Turbine 


Family of curves 


3. Cost: 


All 


u (x ) = (18.5-x )/18.5 
3 3 3 



2. Attribu te P ro perties 



If the attribute properties of order one utility 
independence and pairwise preferential independence hold for 
the attributes cost, schedule and usefulness, then the 
utility function can be expressed in the multiplicative 
form'. Further, if the property of pairwise marginality also 
holds, then the utility function can be expressed in the 
additive form. These forms are shown in Chapter III. 

Verification of utility independence for the 
attributes cost, schedule and future usefulness of the Hark 
92/2 FCS problem is first considered. To verify utility 



independence, consider the attribute cost of the Mark 92/2 
PCS installation at the Combat System LBTS . The minimum and 
maximum costs were determined to be 7.5 million dollars and 
18.5 million dollars. The following question is asked [Ref. 
15]. 



"Suppose the values for future usefulness and 
schedule for the fire control system are specified at some 
given guantity, say, three and 1.5 months. Now, consider 
the gamble 7.5 million dollars with a probability of p and 
18.5 million dollars with a probability of 1-p. Determine a 
dollar amount X such that if you had to choose either the 
gamble or the X million dollars for sure, you would be 
indifferent. Now, suppose I give you a different set of 
values for the other attributes, say, two and fifteen 
months, and ask you to assess X again. Does the value of X 
million dollars change?" 



If the value of X million dollars does not change no matter 
what values are given for the usefulness and schedule 
attributes, and if this is true for all gambles on the cost 
attribute, then the cost attribute is utility independent of 
the other attributes. Utility independence of the 
usefulness and schedule attributes can similarly be verified 
[Ref. 15]. If all of the attributes are utility independent 
of the other attributes, then order one utility independence 
is established. 

1c verify pairwise preferential independence, 
consider the attributes of cost and future usefulness for 
the gas turbine problem. The following dialogue is held 
[Ref. 15]. 



"Consider the consequence I, involving a four 
million dollar installation and testing cost for the gas 
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turbine engine, no future usefulness and an eleven month 
schedule. Now, determine a dollar figure X such that 
conseguence II defined as; X million dollars, a maximum 
future usefulness value of three and an eleven month 
schedule, is exactly as attractive as conseguence I. now, 
let's change the value of the schedule attribute. Does the 
value of X million dollars change?" 



If the value of X million dollars does not change fcr all 
values of the schedule attribute, and if this is true for 
all trade-offs between cost and usefulness, then these two 
attributes are pairwise preferentially independent of the 
schedule attribute. Pairwise preferential independence 
between the ether pairs of attributes is determined in a 
similar manner [Ref. 15], 

To verify pairwise marginality, a convenient test is 
the pairwise marginality test [Ref. 15]. As an 
illustration, the attributes of cost and schedule for the 
SPS-49 radar consequences are considered. Let lottery L 

yield 3.5 million dollars and a schedule of 1.5 months with 

a probability of 0.5 and 3.5 million dollars and a schedule 
of four months with a probability of 0.5. Let lottery L^ 

yield 3.5 million dollars and a schedule of four months with 

a probability of 0.5 and 1.5 million dollars and a schedule 
of 1.5 months with a probability of 0.5. If the 

decision-maker is indifferent between L and L , then 

1 2 

pairwise marginality holds between cost and schedule. 

Similarly, the pairwise marginality tests are made between 
all pairs of the attributes. 

With the information gathered from interviews at the 
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Patrol Frigate Project Office, it was not unreasonable for 
the authcr tc verify the properties of utility independence, 
pairwise preferential independence and pairwise marginality. 
It was determined that the properties of order one utility 
independence and pairwise preferential independence hold for 
the attributes. Pairwise marginality does not hold. This 
result was the same for the three subsystems under analysis. 
By Keeney's Theorem [Ref. 13], the utility functions fcr all 
three subsystem problems can be represented in the 
multiplicative form 

3 

1 + K u (x ,x ,x ) = .TT [1 + Kk.u.(x.)] 

123 1=1 li l 



where x ,x ,x represents the 
12 3 

usefulness, schedule and cost 
function of a single attribute x 

i 



attributes of future 

respectively. The utility 

is represented by u (x ) . 

i i 



In the assignment of utility numbers to 

consequences, the individual attribute utility functions 

have been determined and the utility function, which is a 

combination of the individual utility functions, can be 

expressed in the multiplicative form. What remains is to 

determine the scaling constants K, k , k and k so that the 

12 3 

multiplicative utility function can be expressed in 
numerical terms. This determination is made in the section 
below. 

3 . As ses s men t of- Utility Function Parameters 



A procedure for evaluating the scaling constants, 
when the utility function is in the multiplicative form, is 
found in Ref. 13 and Ref. 15. The basic idea is to define 
four linearly independent equations, since there are four 
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unknowns, K, k , k and k to be evaluated [Ref. 151. The 

12 3 

following scheme is a convenient method for obtaining the 
linearly independent equations. 



Consider the outcomes described by three attributes 

x . As a first step, the decision-maker is asked to rank 
i 

the attributes in order of preference. For illustration, he 



ranks the attributes x such that x > x > x . The utility 

i 12 3 

values m will next be determined from five outcomes. Let 
i 



outcome X^ represent the best possible set of attributes 
* * * 

(a ,b ,c ) . Let outcome X represent the worst possible set 

5 

* * # 

of attributes (a ,b ,c ) . Then u (a ,b ,c ) = 1 and 

* * * 



u (a ,b ,c ) = 0. Let outcome X represent the set of 
♦ £ * 2 



attributes where the worst value for the least preferred 

* 

attribute / c , replaces c in the attribute set of outcome 
* 

* * 



X 

~1 


such 


that X 

~2 


= (a ,b ,c ) . Similarly, let outcome X 
* 3 


represent 


the set 


of attributes where the 


worst value for 


the 


next 


least 


preferred attribute, b , 

* 


* 

replaces b , such 


that 


X = 
“3 


* 

(a ,b , 
* 


* 

c ) . Using the same 


conventions, let 



* * 



outcome X 
" r 4 



(a /b , c )• 
* 
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To assess the utilities of the outcomes X , X and 

”2 ”3 

X , the decision-maker is asked to determine the 

”4 

probabilities of the gambles in the following situations 
where he is indifferent between: 



(a) X for sure and the gamble <X ,p ,X >, 
2 ”12 5 



(b) X for sure and the gamble <X ,p ,X > and 
3 ”1 3 "5 



(c) X for sure and the gamble <X ,p ,X >. 
”4 ”14 ”5 



Cnee the probabilities p^, p^ and p^ are assessed, 

then 



U<i 2> 


■ P 2 M V * 


n-p 2 ) 


U(X ) , 

D 


MX > 


= p u (X ) t 


(i-p j 


u (X ) and 


3 


3 1 


3 


”5 


u <y 


= p u(X ) + 

4 1 


<i-p 4 ) 


MX ) . 
5 



Since u(X ) =1 and u (X ) = 0, it follows that 

1 5 



u (X ) = p = m , 

”2 2 2 

u (X ) = p = m and 

"3 3 3 

u (X ) = p = m . 

“4 4 4 



The utility values m , m and m can be verified for 

12 3 

consistency by asking the decision-maker to assess other 
gambles consisting of various combinations of the five 
outcomes. If the decision-maker has been consistent, then 
the utility values derived should be similar to the 
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previously derived utility values m^, m^ and m^. If they 

are not similar values, the decision-maker is asked to 

re-assess all gambles until some consistency is achieved. 
Table V illustrates the above steps. 



TABLE V 



ILLUSTRATION OF SCALING CONSTANT EVALUATION 



OUTCOME 


ATTRIBUTES 

XXX 

1 2 3 


UTILITY OF OUTCOME 




* * * 




X 


a b c 


m =1 


~1 




0 




* * 




X 


a b c 


m 


~2 


* 


1 




* * 




X 


a b c 


m 


~3 


* 


2 




* * 




X 


a b c 


m 


~ 4 


* 


3 


X 


a b c 


m =0 


"5 


* * * 


4 



Once the utility value m for each of the five 

j 

outcomes are defined, the scaling constant K can be 

determined from four linearly independent equations. Keeney 
[Ref. 13] shows the following relationship holds for the 
multiplicative form: 

3 

1 + K = .TJ (1 + Kk.) (3) 



and, generally [Ref. 13]; 
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1 + m K = fl (1 + Kk ) / (1 + Kk ) . 
i i=l i -i 



<<») 



Equation (3) and equation (4) for j = 1,...,3 are used as 
the four linearly independent equations. From (3) and (4) , 



1 + K = . (1 + Kk.) 



= (1 + m K) (1 + Kk ) . 

3 3 



(5) 



Then from (3) , 



and 



(1 + K) 3 = TT (1 + m K) (1 + Kk ) 
j=i 3 3 

3 3 

= .TT (1 + m .K) Jl (1 + Kk .) . 
3=^ 3 3=' 3 



(1 + k) 3 = TT (1 + m K) (1 + K) 
1=1 1 



(1 + K) 2 = (1 + m K) . 



( 6 ) 



Eguation (6) is a cubic equation in the form, 

K (aK 2 + bK +c) =0. 

Thus, one of the values of the cubic equation is zero. The 
other two values are: 

1/2 

[-b ± (b 2 - 4ac) ] / 2a. 

Keeney [Ref. 13] proves that the range for the values of K, 
will be K > -1 where K ^ 0. Therefore, that value of K 
which is non-zero and greater than -1 is chosen. Giaugue 
[Ref. 12] proves that there is always one and only one such 
value. 
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Once the constant K 

constants k , k and k are 
12 3 



is known, the remaining scaling 
derived from eguation (5) , 



1 + K 



(1 + m K) (1 + Kk ) ; 

j j 



Hence, 



k = (1 - m ) / (1 + m K) . (7) 

3 j 3 



Appendices A, B and C outline the steps and 
computations made to determine the scaling constants for the 
fire control system, the SPS-49 radar and the gas turbine 
engine problem utility functions. The scaling constants are 
listed in Table VI. 



TABLE VI 

SCALING CONSTANTS 



Scaling Constants 


FCS 


Radar 


Gas Turbine 


K 


0.07 


-0.52 


1.37 


k 

1 


0.21 


0.19 


0.07 


k 


0.42 


0.34 


0.26 


2 






- 


k 

3 


0.64 


0.68 


0.44 



The utility functions, using the scaling constants 
from Table VI, are: 

(a) for the Mark 92/2 FCS problem; 

1+0.07u(2) =[ 1 + O.OIu (x ) ][ 1 + 0.03u 2 (x^ ][ 1+0. 05 u^ (x^) ], 

(b) for the AN/SPS-49 radar problem; 
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1-0.52u(_X)=[1-0.1u^(x^) ][1-0.18u_^(x^) ][ 1-0. 36u^ (x^) ], 



and 



(c) for the LM 2500 gas turbine engine problem; 

1 + 1 .37u (X) =[ 1+0. 1u^ (x^) ][ 1 + 0.35u 2 (x^) ][ 1+0. 6u^ (x^) ]. 



The utility of x^, u^ ( x ^ ) , represents the utility of the 

attribute future usefulness, u (x ) represents the utility 

2 2 

of the attribute schedule and u^ (x^) represents the utility 
of the attribute cost. 



The second step in the decision analysis process has 
been completed. Utility functions in the multiplicative 
form have been derived so that utility numbers can be 
expressed for each of the consequences indicating the 
decision-maker's preferences for these consequences. To 
complete formulation of the LBTS problem, judgmental 
probabilities are assessed for the uncertain events of time 
to accomplish tests. Certification, Operational Assist and 
DS ARC III. 

C. ASSESSMENT OF JUDGMENTAL PROBABILITIES 

By informal correspondence, the PF Assistant Test and 
Evaluation Manager was asked to assess the probabilities for 
the uncertain events for the installation and testing cf the 
three subsystems under analysis. Table VII summarizes the 
results of his reply. The probabilities indicated in this 
table are assigned to the appropriate chance forks cf the 
three decision trees. 
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TABLE VII 



JUDGMENTAL PROBABILITIES FOR LBTS PROBLEM 



A. I f ac tual e q uipme n t instal led : 



1 . 

3. 

4. 



5. 



6 . 



B. 



1 . 

2 . 

3. 

4. 



5. 



6 . 



Probability of: 


MK 


92 FCS 


S PS-49 


LH 2500 ENG 


sufficient time for 
Certification 




0.4 


0.8 


0.01 


passing Op Assist 

given passed Certification 


0.7 


0.7 


O 

• 

CD 


passing Op Assist 
given no Certification 
done 




0.2 


C.6 


0.3 


passing DSARC III if 
passed Certification 
and Op Assist 




0.8 


0.7 


0.8 


passing DSARC III if 
passed Op Assist but 
no Certification done 




0.1 


0.5 


0.2 


If part of system simulated: 








Probability of: 


MK 


92 FCS 






sufficient time for 
Certification 




0.4 






passing Certification 
if time available 




0.5 






passing Op Assist 

given passed Certification 


0.4 






passing Op Assist 
given no Certification 
done 




0.2 






passing DSARC III if 
passed Certification 
and Op Assist 




0.6 






passing DSARC III if 
passed Op Assist but 
no Certification done 




0.1 
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C . I f all of syst em sim u lat ed : 



Probability of: 


MK 


9 2 FCS 


SPS-49 


LM 2500 ENG 


sufficient time for 
Certification 




0.2 


0.7 


0.4 


passing Certification 
if time available 




0.3 


0.7 


0.35 


passing Op Assist 

given passed Certification 


0.2 


0.7 


0.4 


passing Op Assist 
given no certification 
done 




0.1 


0.6 


0.2 


passing DSARC III if 
passed Certification 
and Op Assist 




0.1 


0.5 


0.5 


passing DSARC III if 
passed Op Assist but 
no Certification done 




0.05 


0.4 


0.5 



D . I f- s ystem w e re no t installed a t LB TS : 

Probability of: MK 92 FCS SPS-49 

1. passing DSARC III 0.2 0.6 

D. SUMMARY OF CHAPTER 

/ 

The systematic process of decision analysis was used to 
formulate the problem of deciding whether to install actual 
subsystems, partially simulate them or simulate the entire 
subsystem at the Patrol Frigate LBTS. The alternative of 
not installing the subsystem at the LBTS was also 

considered. Three subsystems were chosen from the Risk 
Watch List for the analysis. They are the Mark 92 mod 2 
fire control system, the AN/SPS-49 radar and the LM 2500 gas 
turbine engine. In section A. of this chapter, the 
structure of the problem was formulated in terms of a 
decision tree. Figure 6 illustrates a typical branch of the 
decision tree for the installation of actual equipment, 
partially simulated equipment or simulated equipment. 
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Figure 7 illustrates the "no installation" alternative 
branch of the decision tree. Section B. contained a 
discussion of the second step in the decision analysis 
process, assessment of the decision-maker * s preferences for 
the consequences. They are expressed in terms of utility 
numbers. A utility function which assigns the utility 
numbers to the consequences was formulated in the 
multiplicative form. The equations derived are shown in 
this section. Section C. comprised the third step in the 
decision analysis process, assessment of the judgmental 
probabilities for each chance fork of the decision tree. 
Table VII lists these probabilities. 

Chapter V contains a discussion of the final step in the 
decision analysis process, determination of the best courses 
of action using the information formulated in this chapter. 
The word "final" implies that the process is completed when 
the best course of action is determined. As was pointed 
out, the decision analysis process is iterative. This 
thesis represents the initial analysis of a complicated 
problem. In the conclusion. Chapter VI, some suggestions 
for further research to refine the analysis of this thesis 
are made. 
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V 



• PF LBTS PROBLEM SOLUTION RESULTS 



All of the information required to solve the PF LBTS 
problem, including a discussion of the alternatives and 
conseguences for each subsystem, and cost, schedule and 
usefulness considerations have been provided in the 
preceeding chapter. The functions which transform the three 
attributes into a scalar measure of utility for the outcomes 
were derived, and judgmental probabilities were assessed for 
each uncertain event of the decision tree. All that remains 
is to structure the decision tree, assign the appropriate 
utility to each end point, insert the judgmental 
probabilities at the chance forks and determine the expected 
utility values for each alternative of the tree. 

in this chapter, the solution results for the AN/SPS-49 
radar, the Mark 92/2 FCS and the LM 2500 gas turbine 
problems will be discussed separately. In each case, the 
decision tree is provided before the results are discussed. 
The decision trees presented are analyzed by computing the 
expected utility of each chance fork and the selection of 
the greatest utility at each decision fork. The process is 
repeated for each level of the tree until the starting 
decision fork is reached. The alternative with the greatest 
expected utility is selected as the best course of action. 
A computer program written in the BASIC language, was used 
to perform the calculations. 



A. SPS-49 RADAR 



The decision tree for the AN/SPS-49 radar problem is 
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exhibited in Figure 8. The alternatives which were 

considered are: installation of the actual equipment, 

installation of simulated equipment or no installation. The 
assessed judgmental probabilities are indicated near each 
"Pass" and "Fail" of the chance forks. Table VIII lists the 
outcomes for each of the twenty-six consequences and 

provides the utility for each consequence. The utility 
functions derived for the AN/SPS-49 radar problem are shown 
in section B.3. of the previous chapter. 

The solution results are: 

( 

(D the expected utility of the alternative, 
installation of the actual equipment, is 0.76, 

(2) the expected utility of the alternative, simulation 
of the equipment, is 0.72 and 

(3) the expected utility of the alternative, no 
installation, is 0.81. 

These results indicate that the recommended course of action 
is not to install the SPS-49 radar at the Combat System 
LBTS. 
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DECISION TREE SPS-49 RADAR 
ACTUAL EQUIPMENT AND NO EQUIPMENT BRANCHES 



Consequence 

numEer 




FIGURE 8 
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DECISION TREE SPS-49 RADAR 
SIMULATE BRANCH 



Consequence 

numEer 
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TABLE VIII 



OUTCOMES AND RESULTS OF SPS-49 DECISION TREE 



CONSEQUENCE 
Number 1 


OUTCOMES 

Usefulness Schedule 
(scalar) (month) 


Cost 

(million) 


, UTILITY 


1 


3 


1.5 


2.5 


0.93 


2 


0 


4 


3.5 


0.55 


3 


3 


4 


3.5 


0.68 


4 


0 


4 


3.5 


0.55 


5 


3 


4 


3.5 


0.68 


6 


0 


4 


3.5 


0.55 


7 


3 


4 


3.5 


0.68 


8 


3 


4 


2.5 


0.72 


9 


0 


4 


3 .5 


0.55 


10 


3 


4 


3.5 


0.68 


11 


0 


4 


3.5 


0.55 


12 


3 


4 


3.5 


0.68 


13 


0 


0 


0 


0.90 


14 


3 


4 


3.5 


0.68 


15 


1 


1.5 


1 


0.92 


16 


0 


4 


2 


0.60 


17 


1 


4 


2 


0.67 


18 


0 


4 


' 2 


0.60 


19 


1 


4 


2 


0.67 


20 


0 


4 


2 


0.60 


21 


1 


4 


2 


0.67 


22 


1 


4 


1 


0.70 


23 


0 


4 


2 


0.60 


24 


1 


4 


2 


0.67 


25 


0 


4 


2 


0.60 


26 


1 


4 


2 


0.67 



SOLUTION RESULTS 



ALTERNATIVE 


EXPECTED UTILITY 


r; Actual 


0. 76 


2. Simulated 


0.72 


3. None 


0.81 
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B 



MABK 92 MOD 2 FCS 



The decision tree for the Mark 92/2 FCS problem is 
exhibited in Figure 9. The alternatives which were 
considered are: installation of the actual equipment, 
partial simulation of the equipment, complete simulation of 
the equipment and no installation. The assessed 
probabilities are indicated near each "Pass" and "Fail" of 
the chance forks of the tree. Table IX lists the outcomes 
for each of the consequences and provides a range of 
utilities for each consequence. Recall that a family of 
utility curves was established for the schedule attribute. 
Hence, for the fire control system problem, four different 
sets of points for the schedule attribute were used to check 
the sensitivity of the shape of these curves on the 
solution. The utility functions derived for the fire 
control system installation problem are shown in section 
B.3. of the previous chapter. 

The solution results are: 

(1) the expected utilities for the alternative, 
installation of the actual equipment, range from 0.38 to 
0.63, 



(2) the expected utilities for the alternative, partial 
simulaticn of the equipment, range from 0.50 to 0.78, 

(3) the expected utilities for the alternative, 
simulaticn cf the equipment, range from 0.40 to 0.70 and 

(4) the expected utilities for the alternative, no 
installation, range from 0.43 to 0.67. 

These results indicate that the recommended course of action 
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is to partially simulate the fire control system to be 
installed at the Combat System LBTS. They are consistent 
over the entire range of the utilities. The different 
shapes of the schedule attribute utility curve do not affect 
the solution results. 

C. LM 2500 GAS TURBINE ENGINE 

The decision tree for the gas turbine engine problem is 
shown in Figure 10. Two alternatives were considered. They 
are; installation of the engines or simulation cf the 
engines. The assessed judgmental probabilities are 
indicated near each "Pass" and "Fail" of the chance forks of 
the tree. The outcomes for each of the twenty-four 
conseguences and a range of utilities for the outcomes are 
provided in Table X. As with the fire control system 
problem, a family of curves was established to represent the 
utility curve of the schedule attribute. The range of 
utilities represent four different sets of points used to 
check the sensitivity of the shape of these curves on the 
solution results. 

The solution results are; 

(D the expected utilities of installing the gas turbine 
engines range from 0.19 to 0.35 and 

(2) the expected utilities of simulating the gas turbine 
engines range from 0.4 to 0.64. 

The recommended course of action is to simulate the gas 
turbine engine at the Propulsion System LBTS. These results 
are consistent over the entire range of the utilities. The 
different shapes of the schedule attribute utility curve do 
not affect the solution results. 
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DECISION TREE MARK 92/2 ECS 
ACTUAL EQUIPMENT AND NO EQUIPMENT ERANCHES 



Consequence 

numter 




FIGURE 9 
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DECISION TREE MARK 92/2 FCS 
PARTIALLY SIMULATE BRANCH 



Consequence 

numEer 
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DECISION TREE HARK 92/2 FCS 
SIMULATE BRANCH 



Consequence 

numEer 
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TABLE IX 



CONSEQUENCE NUMBER, OUTCOMES AND RESULTS 
FCS DECISION TREE 



NR 


OUTCOMES 


UTILITY 




Use 


Sched 


Cost 


1 


2 


3 


4 


1 


3 


1.5 


13. 5 


0.81 


0.81 


0.81 


0.81 


2 


0 


1.5 


13.5 


0.42 


0.42 


0.42 


0.42 


3 


3 


6 


18.5 


0.29 


0.42 


0.55 


0.59 


4 


0 


6 


18.5 


0. 08 


0.21 


0.34 


0.38 


5 


3 


6 


18.5 


0.29 


0.42 


0.55 


0.59 


6 


0 


6 


18.5 


0.08 


0.21 


0.34 


0.38 


7 


3 


6 


18.5 


0.29 


0.42 


0.55 


0.59 


8 


3 


6 


13.5 


0.47 


0.60 


0.73 


0.77 


9 


0 


6 


18.5 


0. 08 


0.21 


0.34 


0.38 


10 


3 


6 


18.5 


0.29 


0. 42 


0.55 


0.59 


11 


0 


6 


18. 5 


0.08 


0.21 


0.34 


0.38 


12 


3 


6 


18.5 


0. 29 


0. 42 


0.55 


0.59 


13 


0 


0 


0 


0.99 


0.99 


0.99 


0.99 


14 


3 


6 


18.5 


0.29 


0.42 


0.55 


0.59 


15 


2 


1.5 


9.5 


0.93 


0. 93 


0.93 


0.93 


16 


0 


6 


12.5 


0.74 


0.74 


0.74 


0.74 


17 


2 


6 


12.5 


0.48 


0.60 


0.73 


0.77 


18 


0 


6 


12,5 


0.29 


0.42 


0.55 


0.59 


19 


2 


6 


12. 5 


0.47 


0.60 


0.73 


0.77 


20 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


21 


2 


6 


12.5 


0.47 


0.60 


0.73 


0.77 


22 


2 


6 


9.5 


0.58 


0.71 


0.84 


0.88 


23 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


24 


2 


6 


12.5 


0.47 


0.60 


0.73 


0.77 


25 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


26 


2 


6 


12.5 


0.47 


0. 60 


0.73 


0.77 


27 


1 


1.5 


7.5 


0.92 


0.92 


0.92 


0.92 


28 


0 


1.5 


7.5 


0.81 


0.81 


0.81 


0.81 


29 


1 


6 


12.5 


0.40 


0.53 


0.66 


0.70 


30 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


31 


1 


6 


12.5 


0.40 


0.52 


0.66 


0.70 


32 


0 


6 


12.5 


0. 29 


0.42 


0.55 


0.59 


33 


1 


6 


12.5 


0.40 


0.52 


0.66 


0.70 


34 


1 


6 


7.5 


0.57 


0.7 0 


0.83 


0.88 


35 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


36 


1 


6 


12. 5 


0.40 


0. 53 


0.66 


0.70 


37 


0 


6 


12.5 


0.29 


0.42 


0.55 


0.59 


38 


1 


6 


12.5 


0.40 


0.53 


0.66 


0.70 



SOLUTION RESULTS 



ALTERNATIVE 




EXPECTED 


UTILITY 




1. Actual 


1 

0. 38 


2 

0.48 


3 

0.59 


4 

0.63 


2. Partially simulated 


0.50 


0.62 


0.74 


0.78 


3. Simulated 


0.40 


0.53 


0.66 


0.70 


4. None 


0. 43 


0. 53 


0.64 


0.67 
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DECISION TREE LH 2500 GAS TURBINE ENGINE 
ACTUAL EQUIPMENT BRANCH 



Consequence 

numEer 




FIGURE 10 
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EECISI ON TREE LM 2500 GAS TURBINE ENGINE 
SIMULATE BRANCH 



Consequence 

numcer 
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TABLE X 



CONSEQUENCE NUMBER, OUTCOMES AND RESULTS 
GAS TURBINE DECISION TREE 



NR 


OUTCOMES 


UTILITY 




Use 


Sched 


Cost 


1 


2 


3 


4 


1 


3 


10 


14 


0. 51 


0.51 


0.51 


0.51 


2 


0 


16 


15 


0.11 


0. 14 


0.20 


0.25 


3 


3 


16 


15 


0. 19 


0.22 


0.29 


0.35 


4 


0 


16 


15 


0.11 


0. 14 


0.20 


0.25 


5 


3 


16 


15 


0.19 


0.22 


0.29 


0.35 


6 


0 


16 


15 


0.1 1 


0. 14 


0.20 


0.25 


7 


3 


16 


15 


0. 19 


0.22 


0.29 


0.35 


8 


3 


16 


14 


0.22 


0.25 


0.31 


0.38 


9 


0 


16 


15 


0.11 


0. 14 


0.20 


0.25 


10 


3 


16 


15 


0. 19 


0.22 


0.29 


0.35 


1 1 


0 


16 


15 


0. 1 1 


0. 14 


0.20 


0.25 


12 


3 


16 


15 


0.19 


0.22 


0.29 


0.35 


13 


1 


10 


3 


0.82 


0.82 


0.82 


0.82 


14 


0 


16 


4 


0.38 


0. 42 


0.50 


0.57 


15 


1 


16 


4 


0.43 


0.47 


0.55 


0.63 


16 


0 


16 


4 


0.38 


0.42 


0.50 


.57 


17 


1 


16 


4 


0.43 


0.47 


0.55 


0.63 


18 


0 


16 


4 


0.38 


0.42 


0.50 


0.57 


19 


1 


16 


4 


0.43 


0.47 


0.55 


0.63 


20 


1 


16 


3 


0.46 


0.50 


0.58 


0.66 


21 


0 


16 


4 


0.38 


0.42 


0.50 


0.57 


22 


1 


16 


4 


0.43 


0.47 


0.55 


0.63 


23 


0 


16 


4 


0.38 


0. 42 


0.50 


0.57 


24 


1 


16 


4 


0.43 


0.47 


0.55 


0.63 



SOLUTION RESULTS 



ALTERNATIVE 




EXPECTED 


UTILITY 






1 


2 


3 


4 


Actual 


0. 19 


0.23 


0.29 


0.35 


Simulated 


0.45 


0. 49 


0.56 


0.64 
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VI. SUMM ARY AND RECOMM ENDATIO NS 



In this thesis, decision analysis methodology was 
applied to the problem of some project management trade-off 
decisions required for the Patrol Frigate Land Based Test 
Sites. In particular, the question of what PF subsystems 
should be installed and tested at the LBTS's was studied. 

This chapter summarizes the content of the thesis, 
presents recommendations and provides suggestions for 
further research. 

A. THESIS SUMMARY 



Chapter II provided the background of the Patrol Frigate 
Land Based Test Sites. The DSARC review process was 
introduced along with the required test and evaluation 
requirements necessary in each stage of the Patrol Frigate's 
acquisition cycle. The problem of what the composition of 
subsystems should be in the Combat and Propulsion System 
LBTS so that adequate test and evaluation with satisfactory 
results can be supplied to DSARC III, was provided as an 
example to which decision analysis methodology can be 
applied. 

Chapter III supplied the decision analysis methodology 
reguired to formulate and solve the PF LBTS problem 
described in Chapter II. The steps in the formal process of 
decision analysis were discussed. They are: 

(1) structuring the problem in terms of a decision tree. 



82 



(2) evaluation of the consequences in utility numbers. 



(3) assignment of probabilities to the branches of the 
chance forks of the decision tree and 

(4) determination of the optimal strategy by the 
"averaging out and folding back" process. 

It was pointed out that the formal analysis is an iterative 
process. First, a broad description of the problem with 
rough assessments of the utilities and probabilities is made 
to determine whether any branches at decision forks can be 
eliminated or added. Next, the measurements are refined and 
certain forks added or eliminated. This cycle is repeated 
until there is satisfaction with the results of the 
analysis. . 

The decision tree represents the chronological 
arrangement of the choices or alternatives controlled by the 
decision-maker and the choices determined by chance. The 
evaluation procedures for the assignment of utility numbers 
to each conseguence were discussed. A consequence is 
represented by a path through the complete decision tree. 
The utility numbers are generated by a utility function 
which transforms or maps the attribute describing the 
outcome of a consequence into a scalar value. The 
conseguences for a complex problem are generally described 
by multiple attributes which complicate the construction of 
the utility function. Simplification of the multiattribute 
utility function was discussed utilizing the properties of 
utility independence, pairwise preferential independence and 
pairwise marginality. Table I summarized the properties 
necessary for each type of utility function simplification. 
The assessment of the judgmental probabilities tc each 
chance fork of the decision tree was. then outlined. With 
all the necessary information available for the decision 
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tree analysis, the technique of "averaging out and folding 
back" to determine the expected utility of each alternative 
was described. The alternative with the greatest expected 
utility is chosen as the recommended course of action. 

The Patrol Frigate LBTS problem was formulated in 
Chapter IV. This formulation encompassed the initial 
iteration of a formal decision analysis process. The Mark 
92/2 fire control system, the AN/SPS-49 radar and the LM 
2500 gas turbine engine were the PF subsystems examined. In 
the testing and evaluation process of these subsystems at 
the LBTS, the Project Manager worried about the installation 
and testing costs, the schedule for adequate test and 
evaluation, the uncertainty of passing DSARC III review and 
the usefulness of the subsystem at the LBTS to the program 
after DSARC III. His alternatives, in general, were to: 

(1) install actual or simulated equipment, at the LETS, 

(2) conduct at-sea testing of each subsystem during 
Operational Evaluations, 

(3) monitor tests at the vendor's activity and at sea or 

(4) conduct no further testing. 

In passing DSARC III, the Project Manager is concerned with 
the uncertainty of passing Certification and Operational 
Assist prior to the DSARC III review. He is also uncertain 
as to the availability of time to accomplish these test 
requirements. These uncertainties were incorporated into 
the decision tree structure as chance forks in the 
appropriate order along the paths of the tree. The 
alternatives chosen for the fire control system were to 
install the actual equipment, partially simulate the fire 
control system, simulate all o£ the eguipment or not to 
include the fire control system at the Combat System LBTS. 
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The alternatives chosen for the AN/SPS-49 radar were to 
install the actual equipment, simulate the radar system or 
not to include the radar at the Combat System LBTS. Two 
alternatives were chosen for the LM 2500 gas turbine engine. 
They were to install the engine or simulate the engine at 
the Propulsion System LBTS. 

Cost, schedule and future usefulness were selected as 
the attributes describing the outcomes of each conseguence 
of the tree. These attributes for the three subsystem 
problems were determined to have the properties of first 
order utility independence and pairwise preferential 
independence. As a result, the utility functions were 
expressed in the multiplicative form. The judgmental 
probabilities assessed for each chance fork are listed in 
Table VII. 

With the decision tree structure defined, the utility 
functions derived and the chance fork probabilities 
assessed, each subsystem problem was structured in terras of 
a decision tree in Chapter V. A computer program, written 
in the EASIC language, was constructed to evaluate the 
conseguences of the tree in utility numbers and to "average 
out and fold back" the tree to arrive at the expected 
utility of each alternative considered. Tables VIII, IX and 
X list the results. For the SPS-49 radar, the recommended 
alternative was not to install the radar at the Combat 
System LBTS. For the LM 2500 gas turbine engine, the 
recommended alternative was to simulate the engine at the 
Propulsion System LBTS. For the Mark 92/2 FCS, the 
recommended alternative was to partially simulate the fire 
control system. 
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B. RECOMMENEATIONS 

The decision tree analysis yielded several 
recommendations which contrast with the alternatives 
selected by the PF Project Manager. In actuality, it has 
been determined that the SPS-49 radar and the LM 2500 gas 
turbine are to be installed at their respective Land 3ased 
Test Sites [Ref. 5]. A result of this thesis is that the 
radar should not be installed and the gas turbine engine 
should be simulated. It is recommended that these 
alternatives should be further considered by the PF Project 
Manager. In the case of the SPS-49 radar, it is not too 
late to consider the alternative of not installing the 
subsystem at the LBTS. Installation of the LM 2500 gas 
turbine engines is already in progress and it is, perhaps, 
too late to consider its simulation. The alternative 
selected for the Mark 92/2 FCS was to partially simulate the 
fire control system which is in agreement with what was 
actually decided. This alternative, however, was primarily 
selected by the Project Manager because of the location of 
the Combat System LBTS where, out of necessity, the missile 
launchers and gun systems have to be simulated [Ref. 5]. 

The alternative of not installing the LM 2500 gas 
turbine at the LBTS was not considered in the analysis. If 
there is no engine, then there is no propulsion system. 
This alternative, if considered, really leads tc the 
guestion of the desirability of a propulsion system LBTS. 
Rough data was available for consideration of the 
alternative of installing no engine and the expected utility 
of this alternative was computed. It was found that the 
expected utility was greater than that of the other two 
alternatives considered. This result would lead directly to 
the recommendation that a Propulsion System LBTS would not 
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be required. It is recommended that for future ship 
acquisition projects, the desirability of a propulsion 
system LETS be closely studied. A propulsion system LBTS 
should not be preordained by the spirit of existing DOD 
Directives. 

The analysis of this thesis has shown some results which 
are in contrast to the alternatives actually selected. This 
thesis has shown that decision analysis can be a viable 
decision-making tool for the project manager. It is a 
stimulus to think about new alternatives. It allows the 
decision-maker to scrutinize a problem as an organic whole. 
Decision analysis provides the decision-maker with the means 
to distinguish his preferences for consequences frcm his 
judgments abcut uncertainties [Ref. 11]. Decision analysis 
also allows the project manager to demonstrate that his 
decisions were not made on a casual basis and that a 
formalized decision approach was used. He can use the 
analysis to communicate the rationale of the chosen strategy 
to obtain support for his decision. It is recommended that 
decision analysis methodology be incorporated more fully 
into the project management environment. Raiffa [Ref. 11] 
states : 



"In my personal opinion. one part of the 
justification for adoptina the methodology of decision 
analysis is that the underlying behavioral assumptions 
are appealing; a second part of the justification is 
that this metnodology is an operational mode of analysis 
(at least for many problems, and the class is widenina) : 
and the final part of the justification is, what would 
you do otherwise?" 



C. SUGGESTIONS FOR FURTHER RESEARCH 



This thesis represents an initial analysis cf a 
complicated problem. Further research is necessary to 
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refine the probability and utility measurements. In 
particular, the effect of performance as an attribute should 
be further studied. This is a very complicated area and 
would make a thesis in its own right. The probability 
assessments were made at the project manager level. These 
probabilities could be further refined by assessing the 
judgments of technical personnel located at the L3TS and at 
the level of the Office of the Chief of Naval Operations. 
These assessments could then be compared and refined for the 
next iteration of the analysis. 

The tools developed in this thesis can be applied to 
ether areas of system acquisition program management. Some 
suggested areas are: 

(D an analysis of procurement contract alternatives of 
a major weapons system acquisition, 

(2) an analysis of the different options in the 
selection of subsystems necessary for a ship or weapons 
system, 

(3) an analysis of the alternatives among different 
types of hull platforms for a designated weapons system such 
as the AEGIS system and 

<<*) an analysis of a centralized procurement plan for 
material other than government furnished material versus the 
alternative of independent procurement of materials by 
several contractors for the same weapons system acquisition 
project. 



D. A CONCLUDING REMARK 



It is hoped that this thesis has generated some interest 
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in decision analysis in the project management environment. 
The decisions required are complex and involve a great deal 
of uncertainty in an area where cost and schedule over-runs 
make it desirable that the option selected is the best 
possible. The Congress and concerned citizens demand it. 
Decision analysis is a tool that managers in the procurement 
of major weapons systems can use. The ideas, of decision 
analysis are relatively new but they are too promising to 
ignore. Decision analysis can be experimented with by 
applying it to selected problems on paper and comparing the 
results with decisions that are actually made. If the 
results are favorable or raise some valid questions not 
considered, as in the example provided by this thesis, then 
these techniques should be adopted for use in the future. 
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APPENDIX A 



S CALING CONS TANT EV ALUATI ON MARK 92 /2 FCS 



The steps in evaluating the parameters of the utility 
function for the Mark 92/2 FCS decision tree are outlined in 
this appendix. This involves evaluation of the scaling 



Section E.3., Chapter IV, contains a description of this 
process. 

A. PREFERENCE RANKING OF ATTRIBUTES 

The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 
schedule, and schedule is preferred to usefulness [Ref. 19]. 

B. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 



constants K, k 



, k and k of the utility function 
12 3 



3 



1 




+ Kk.u. (x.) ] 



i i i 



To establish four linearly independent equations. 



equation (3) , Chapter IV, 



3 

(1 + Kk ) 
i 




and eguation (4) 




+ Kk .) , 

3 

1 , . , 3 , is f irst 



are used. The utility value m , where j = 1 

3 
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determined by the assessment of gambles of the alternatives 



listed below. 



ALTERNATIVES 


USEFULNESS 

(scalar) 


SCHEDULE 

(month) 


COST 

(million) 


A 


3 


1.5 


7.5 


B 


0 


1.5 


7.5 


C 


3 


6 


in 

• 


D 


3 


1.5 


18.5 


E 


0 


6 


in 

• 

CD 



The gambles are assessed per the following table: 





GAMBLE 






UTILITY (m ) 

j 


: is 


indifferent 


to 


<A,.7,E> 


m 

2 


= 


0.70 


) is 


indifferent 


to 


<C, .6,E> 


m 

3 




0.44 


I is 


indifferent 


to 


<A, .6 ,E> 


m 

1 


= 


0.88 


: is 


indifferent 


to 


<B, . 5 , D> 


m 

2 


= 


0.66 


i is 


indifferent 


to 


<B, .5,E> 


m 

3 


= 


0.44 


I is 


indifferent 


to 


<A,.8,D> 


m 

1 


= 


0.88 


: is 


indifferent 


to 


<B, . 3,E> 


m 

2 


= 


0.70 


) is 


indifferent 


to 


<A, .4,E> 


m 

3 


= 


0.40 


\ is 


indifferent 


to 


<A, . 9,E> 


m 

1 


= 


0.90 



The utility of B, m^, is selected as 0.88. The utility of 

C , m , is selected as 0.66. The utlity of D, m , is 
2 3 



selected as 0.44. 



91 



C. SOLUTION OF SCALING CONSTANTS 



With 




m and m assessed, 
.2 3 



the scaling constant K 



be determined from eguation (6), Chapter IV, 



3 

(1 + K) 2 = JT (1 + m K) 
j=1 j 

= (1 + .88K) (1+.66K) (1 + .44K). 



Hence, 



K (.26K2+.26K-.02) = 0, 



where K has the values 0, -1.08 and 0.07. Keeney [Eef. 
shows that the value K which is nonzero and greater than 
is chosen, therefore, K = 0.07 is selected. 



Since the constant K is known, k , k and k can 

1 2 3 

determined from eguation (7) , Chapter IV, 



Therefore , 



and 



k = (1 - m ) / (1 + m K) . 

j j j 

k = (1 - .88) / [ 1 + .88 (.07) ] 

1 

= 0 . 21 , 

k = (1 - .66) / [ 1 + .66 (.07) ] 
= 0.42 

k = (1 - . 44) / [ 1 + .44 (.07) ] 

3 

= 0.64. 



can 



13] 

-1 



be 
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APPENDIX B 



SCALING C ONS TAN T E VALU ATI ON SPS-49 RADAR 



The steps in evaluating the parameters of the utility 
function for the SPS-49 radar decision tree are outlined in 
this appendix. This involves evaluation of the scaling 
constants K, k , k and k of the utility function 



Section E.3., Chapter IV, contains a description of this 
process. 

A. PREFERENCE RANKING OF ATTRIBUTES 

The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 
schedule, and schedule is preferred to usefulness [Ref. 19], 

B. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 



1 * 2 



3 



3 



1 




+ Kk u (X ) ] 
i i i 



To establish four linearly independent eguations. 



eguation (3) , Chapter IV, 



3 

1 + K = TT O + Kk . ) 



i=1 ' i 



and eguation (4) 



3 

1 + m K = JT (1 + Kk ) / (1 + Kk ) , 
3 i=1 i 3 



1 + m K 
3 



are used. The utility value m , where j = 1,.,3, is first 

3 
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determined by the assessment of gambles of the alternatives 
listed below. 



ALTERNATIVES 


USEFULNESS 


SCHEDULE 


COST 




(scalar) 


(month) 


(million) 


A 


3 


1.5 


1.5 


E 


0 


1.5 


1.5 


C 


3 


4 


1.5 


D 


3 


1.5 


3.5 


E 


0 


4 


3.5 



The gambles are assessed per the following table: 



GAMBLE 

B is indifferent to <A, 
C is indifferent to <A, 
D is indifferent to <A, 
B is indiffernt to <A,. 
C is indifferent to <B, 
D is indifferent to <B, 
B is indifferent to <A, 
C is indifferent to <B, 
D is indifferent to <C, 



. 9, E> 

• 8,E> 

. 5,E> 

5,c> 

.8,E> 

. 5,E> 

. 8,D> 

. 7 , D> 

. 6,E» 



UTILITY (m ) 

j 



m = 0.90 

1 

m = 0.80 
2 

m = 0.50 
3 

m =0.9 

1 

m = 0.72 
2 

m = 0.45 
3 

m = 0.90 

1 

m = 0.78 
2 

m = 0.48 
3 



The utility of B, m , ' is selected as 0.9. The utility of C, 

1 

m , is selected as 0.8 and the utility of D, m , is selected 



as 0.5. 
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c 



SOLUTION OF SCALING CONSTANTS 



With id , m and m assessed, the scaling constant K can 
12 3 

be determined from equation (6) , Chapter IV, 



Hence, 



3 

(1 + K) z = .TT (1 + m .K) 

3=1 3 

= (1 + .9K) (1+ .8K) (1+.5K) . 



K (.36K2+.57K+.2) = 0, 



where K has the values 0, -1.06 and -0.52. Keeney [Ref. 13] 
shows that the value K which is nonzero and greater than -1 
is chosen, therefore, K = -0.52 is, selected. 



Since the constant K is known, k , k and k can be 

1 2 3 

determined from equation (7) , Chapter IV, 



Therefore, 



and 



k = (1 - m ) / (1 + m K) . 
3 3 3 



k = (1-.9) / [ 1 - . 9 (.52) ] 
= 0. 19, 

* 2 = (1-.8) / [ 1-r.8 (.52) ] 
= 0.34 



k 3 = (1-.5) / [ 1-.5 (.52) ] 

= 0 . 68 . 
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APPENDIX C 



S CALING CON S TANT EVALUAT ION GAS TUREI NE ENGINE 



The steps in evaluating the parameters of the utility 

function for the gas turbine engine decision tree are 

outlined in this appendix. This involves evaluation of the 

scaling constants K, k , k and k of the utility function 

1 2 3 

3 

i + ku (x) = rr c i + Kk u 

i=1 i i i 

Section B.3., Chapter IV, contains a description of this 
process. 



A. PREFERENCE RANKING OF ATTRIBUTES 



The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 
schedule, and schedule is preferred to usefulness [Ref. 19]. 

B. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 



To establish four linearly independent equations, 
eguation (3) , Chapter IV, 



1 + K = 



ill 



(1 + Kk ) 
i 



and eguation (4) 



1 + m K = TT (1 + Kk ) / (1 + Kk ) , 
i i= 1 i i 



are used. The utility value m , where j = 1,.,3, is first 

j 
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determined by the assessment of gambles of the alternatives 



listed below. 



ALTERNATIVES 


USEFULNESS 

(scalar) 


SCHEDULE 

(month) 


COST 

(million) 


A 


3 


10 


3 


B 


0 


10 


3 


C 


3 


16 


3 


D 


3 


1 10 


15 


E 


0 


16 


15 



The gambles are assessed per the following table: 



GAMBLE 

C is indifferent to <A, 
B is indifferent to <A, 
D is indifferent to <A, 
C is indifferent to <B, 
B is indifferent to <A, 
D is indifferent to <C, 
C is indifferent to <B, 
B is indifferent to <A, 
D is indifferent to <B, 



UTILITY (m ) 



5,E> 

7,0 

3 , E> 
5 , D> 
7 , d> 
6 ,E> 
6 , E> 
8 , E> 

4 ,E> 



m 

m 

m 

m 

m 

m 

m 

m 

m 



2 

1 

3 

2 

1 

3 

2 

1 

3 



= 0.5 
= 0.85 
= 0.3 
= 0.58 
= 0.82 
= 0.34 
= 0.51 
= 0.8 
= 0.34 



The utility of B, m^, is selected as 0.85. The utility of 

C, m , is selected as 0.55 and the utility of D, m , is 
2 3 

selected as 0.35. 
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C. SOLUTION OF SCALING CONSTANTS 



With m , m and m assessed- the scaling constant K 
12 3 

be determined from equation (6) , Chapter IV, 



Hence, 



3 

(1 + K)2 = .TT (1 + m.K) 

3 = 1 3 

= (1 + .85K) (1 + .55K) (1 + .35K). 



K (. 16K2-. 04K-.25) = 0, 



where K has the values 0, -1.11 and 1.37. Keeney [Ref. 
shows that the value K which is nonzero and greater than 
is chosen, therefore, K = 1.37 is selected. 



Since the constant K is known, k , k 

1 2 

determined from equation (7) , Chapter IV, 



and k can 
3 



k 

3 

Therefore, 

k 

1 



k 

2 



and 

k 

3 



= (1 - m ) / (1 + m K) . 

3 j 

= ( 1 - . 85) / [ 1+.85 (1.37) ] 
= 0.07, 

= (1-.55) / [ 1 + .55 (1.37) ] 
= 0.26 

= ( 1 - . 35) / [ 1+.35 (1.37) ] 
= 0.44 



can 



13] 

-1 



be 
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